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DECLARATION OF FACT UNDER 37 C.F.R. §1.131 



We, Renee M. Kovales, Edith H. Stern, and Barry E. Willner, hereby declare the following: 

1) We are co-inventors of the invention described and claimed in U. S. Patent Application 
Number 10/632,177 (hereinafter, "the Subject Application"), entitled "Selectable Audio and 
Mixed Background Sound for Voice Messaging System ", filed on July 3 1, 2003. 

2) We are also co-inventors of the invention described in claimed in U. S. Patent Application 
Number 09/782,773 (hereinafter, "the Parent Application"), now U. S. Patent 7,003,083 B2, from 
which the Subject Application claims priority. The Parent Application is also entitled "Selectable 
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Audio and Mixed Background Sound for Voice Messaging System" and was filed on February 
13,2001. 

3) We conceived of the invention — as described and claimed in Claims 1, 9, 11, 14 - 15, 17 - 
18, 20 - 24, 28, 34, 37, 90, and 99 - 1 15 of the Subject Application (hereinafter, "the claimed 
invention") — in this country before November 10, 2000. Our prior conception to practice is 
evidenced by the following: 

Attached hereto as Exhibit A are selected pages of an Invention Disclosure document 
prepared prior to November 10, 2000. These pages serve as the Invention Disclosure for the 
Parent Application, and by virtue of the claim of priority in the Subject Application, also describe 
the invention claimed in the Subject Application. In a first aspect, the claimed invention 
comprises enhancing a voice mail message for playback to a listener by: creating the voice mail 
message for the listener, by a caller using a telephone device, the voice mail message comprising a 
plurality of message segments; identifying, by the caller using the telephone device, at least two of 
the message segments which are to have background sounds associated therewith; selecting, for 
each of the identified message segments by the caller using the telephone device, a sound to be 
associated therewith as the background sound, wherein the sounds selected as the background 
sound for at least two of the message segments are different sounds; and responsive to a request 
from the listener for playback of the voice mail message, playing back at least two of the identified 
message segments for which a sound was selected to be associated therewith as the background 
sound, while concurrently playing back the sound selected as the background sound for each of 
the played-backed identified message segments, and wherein the sounds selected as the 
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background sound for at least two of the played-back identified message segments are different 
sounds, such that the background sound played back for the voice mail message changes at least 
once during the playback. In a second aspect, the claimed invention comprises enhancing voice 
mail messages for playback to a listener by: creating a voice mail message for the listener, by a 
caller using a telephone device; identifying, by the caller using the telephone device while creating 
of the voice mail message, a plurality of message segments for segmenting the voice mail 
message; selecting, by the caller using the telephone device, an audio file to insert into the voice 
mail message between one or more selected pairs of successive ones of the message segments, 
such that the selected audio file is associated with the selected pair(s) of message segments; and 
inserting the selected audio file between the successive ones of the message segments in its 
associated pair(s) as the voice mail message is played back to the listener. (Portions of the 
Invention Disclosure may be redacted to remove information not necessary to establish the 
invention's conception or reduction to practice.) 

4) The invention was diligently reduced to constructive practice in this country from prior to 
November 10, 2000 to the filing of the Parent Application on February 13, 2001, as evidenced by 
the following facts: 

a) The Invention Disclosure document was mailed to a registered patent attorney on 
September 13, 2000 (notably, prior to November 10, 2000), with a target filing date of January 

3 1, 2001, for the purpose of preparing and filing a utility patent application, which led to the 
February 13, 2001 filing of the Parent Application. 

b) Initial discussions of the invention of the Parent Application with the registered 
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patent attorney occurred on December 8, 2000 and December 13, 2000, and continuing into early 
January, 2001. 

c) A first draft of the Parent Application was created by the registered patent attorney 
on January 5, 2001 and distributed to us for review. 

d) Second and third drafts of the Parent Application were also created by the 
registered patent attorney, and distributed to us for review, responsive to our comments on each 
successive draft, in January, 2001. 

e) Following review of the third draft, our comments were sent to the registered 
patent attorney, leading to preparation and distribution of an approval draft of the Parent 
Application on January 31, 2001. 

f) Following review of the approval draft, and our approval thereof, the final draft of 
the Parent Application was sent by the registered patent attorney to IBM Corporation on January 
31,2001. 

g) Following receipt of this final draft of the Parent Application by IBM Corporation, 
we executed an Oath/Declaration therefor on February 12, 2001; February 7, 2001; and February 
7, 2001, respectively, after which the Parent Application was filed by IBM Corporation on 
February 13, 2001. 

5) We hereby declare that all statements made herein of our own knowledge are true and that 
all statements made on information and belief are believed to be true; that these statements are 
made with the knowledge that willful false statements and the like so made are punishable by fine 
or imprisonment, or both, under 18 U. S. C. §1001; and that such willful statements may 
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jeopardize rhc validity oi'lhc appiicaiion or any patent issued thereon. 




Hchih 'A Stern ' """ ' """Hare 
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'Title of disclosure (in English ) 

Selectable Background Sound 



Status 


Final Deasion {Rte) 


Final deadline 


Final deadline reason 


Docket family 


RSW9-2000-0128 


Processing location 


Raleigh RSW 


* Functional area 


(RSW) Horn: Technology Group (Kopkind.Lindquist) 


Attorney/Patent professional 


Jeanim* Ray/RalBigMBM 


Invention development team (IDT) 


Mania .1 tton/F ilei iai L M 


Submitted date 


08/31/2000 04:17:56 PM EDT 


' Owning division AIM 


Incentive program 


Lab 




* Technology code 





Patent value tool (PVT) sc ore 
Inventors with a Blue Pages entry 

Inventors: Renee Kovales/Raleigh/IBM, Barry WiUnerfWatson/IBM, Edie Stem/Fort Lauderdale/IBM 

Inventor Name Serial Otv/Pept Phone Manager Name 

> Kavates, Renee 10/M0FA N/A Pazefsky. D.P. (Diane) 
Willner, Barry E. 7H/M21A N/A ^joer Judy B. 
Stem, Edith H, 32AA7A N/A Harrison, Colin G. 

> denotes primary contact 

Inventors without a Blue Pages entry 

Invention Development Team Information 

Attorney/Patent Jeanine Ray/Raleigh/IBM 
professional 

Invention development Marcia L Stockton/Raleigh/IBM@IBMUS 
team (IDT) 

Response due to IP&L 10/01/2000 



To view the Main Idea of this disclosure, open the "Main Idea" document from the view 
'Critical Questions (Questions 1-9 must be answered in English) 
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Question 1 

On what date was the invention workable? 08/01/2000 Please format the date as MM/DD/YYYY 
(Worka ble mean s i.e. whe n you know that your design will s olve t he problem ) 



'Question 2 

Is there any planned or actual puohcation or disclosure of your invention to # 
anyone outside IBM? 

If yes. Enter the name of each publication cr patent and the date published below. 

Publication/Patent: 
Date Published or Issued. 



If yes, Enter the name of each publication or patent and the date'published below! 

PuijIiuuiiorvFatcnt 
Date Published o r Issued: 



Has the subject matter of the invention or a product incorporating the invention • no 
been sold, used internally in manufacturing, announced for sale, or included in a 
proposal? 

Is a sale, use in manufacturing, product announcement, or proposal planned"? Yes 

_____ 9 No 

"if Yes, identify the product if known and indicate the date or planned date of sale, announcements, c 
proposal and to whom the sale, announcement or proposal has been or will be made. 

Product: 
Version/Release: 



It mom than one, use cut and paste and append as necessa-y in the 'ielii provided. 

"Question 4 y 
Was the subject matter of your invention or a product incorporating your • n 

invention used in public, e.g., outside IBM or in the presence of non-IBMers? 
If yes, give a date. Please format the date as MM/DD/YYYY 



■Question 5 Yes 
Have you ever discussed your in vention with others not employed at IBM? • No 
If yes, identify individuals and date discussed . Fill in the text area with the following^ information, the 
names of the individuals, the employer, date discussed, under CDA, and CDA #. 



'Question 6 

Was the invention, in any way, started or developed under a government 
contract or project? 


Yos 

Not sure 




If Yes, enter the contract number 












'Question 7 

Was the invention made in the course of any alliance, joint development or oth 

contract activities? 

If Yes, enter the following: 


Yes 
ier • No 

Not Sure 
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Title of disclosure (in English ) 

Selectable Background Sound 



1 Describe your invention, stating the problem solved (if appropriate), and indicating the advantages of 
using the invention. 

Reference disclosure RSW8-2Q00-0149 

2. How does the invention solve the problem or achieve an advantage, (a description of "the invention", 
including figures inline as appropriate)? 



3. If the same advantage or problem has been identified by others (inside/outside IBM), how have those 
others solved it and does your solution differ and why ts it better? 



4. If the invention is implemented in a product or prototyoe, include technical details, purpose, disclosure 
details to others and the date of that implementation. 
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Title of disclosure (in English ) 

Audio Content Context and Enhancement 

Main Idea 

1 . Describe your invention, stating the problem solved (if appropriate), and indicating the advantages of 
using the invention. 

Our invention, audio content context and enhancement, is a system and method to provide additional 
context during audio messages, and to provide disambiguation of topics during audio messages . Further 
it allows "skimming" of voice mail analogous to a reader skimming the contents of an e-mail message. 

Face to face communication between people involves many parallel communication paths . We derive 
information from body language, from words, from intonation, from facial expressions, from the distance 
between our bodies. Distance communication generally involves only a few of these paths, with users 
trying to overcome the limitations so imposed . Today, unified messaging and network convergence 
exacerbates the problem by adding the difficulties of media transformation to our communication . 

E-mail tries to overcome these problems by using font sizes, colors and emoticons (combination of text 
symbols which bear a semblance to facial expressions ) to express non-text information. This information 
includes emphasis, emotion, irony, etc. When e mail is transformed via an audio read out, as is done in 
unified messaging, this context can be lost. In email, the paragraph structure and use of holding and 
italics gives clues as to the number and importance of topics . Thus, e-mail tries to overcome the 
limitations of distance by using visual clues for both semantic and contextual meaning . Further, visual 
cues such as paragraph distinctions pertaining to the end of the message on the page are perceivable as 
soon as the page is displayed, providing a "broadside" perception of the message. 

Voicemail has a different set of problems. When equivalent information is left via voicematl, while the 
recipient has the benefit of the nuances available through voice, the recipient does not have the 
advantages of the other parallel forms of communication available in person . Unlike e-mail, with 
voicemail the recipient does not have the advantage of broadside perception of the message . Our 
invention, audio content context and enhancement, helps to overcome both these limitations . 

2. How does the invention solve the problem or achieve an advantage, (a description of "the invention", 
including figures inline as appropriate)? 

Our invention is a method and system for disambiguating top;cs in an audio message and for 
communicating context associated with a message, or part of a message, with an audio cue (eg level of 
certainty of translation may be represented by a meaningful background tone). The invention enables 
rapid navigation of voicemail messages from an ordinary, non-screen, telephone, and rapid retrieval of 
phone numbers from the middle of a voicemail message. This invention also enables messages to be left 
with "favorite music" tags. 

Base level is delineation (disambiguate) 
Next level is selection of context 
Personal choice of audio (theme song) 

Disambiguating topics : 

Leaving a message - Speakers tend to be less precise in oral speech than they are in written 
communications. This can be inefficient when communicating via voicemail , Our invention includes a 
method and system for such speakers to signal, with a keyed alert sequence (KAS) during the course of 
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the message they are leaving, that they are now changing topic, or marking a particularly important 
passage. This is accomplished with a sequence of keystrokes on the telephone keypad . For example, in 
a message covering two topics, the speaker may pause, key in the KAS (e.g., *), and continue speaking. 
The KAS alerts the voicemail system that a new topic has been started. If a very important passage is 
about to be communicated, e.g., a phone number or identifier, the speaker may key in the appropriate 
alert (e.g., " or perhaps another sequence such as *9) before communicating the information 

Note; In the above description, the KAS may be recorded as part of the audio stream. In an alternate 
embodiment, KAS detection causes the development of metadata associated with the audio stream which 
allows the voicemail system to index to the KAS points . 

Preparing the message for retrieval - The voicemail system analyzes each message for KAS tones . If 
such tones have been used, the voicemail system can associate an audio cue with each section of the 
message so delineated. The associated cues are mixed in with the message, i.e., "conferenced". 
Alternatively, the voicemail system may alternate between two such audio cues, with adjacent sections 
each having one of the cues. Such cues may be musical, for example tunes. They may be natural 
sounds such as btrdsong, ocean waves. One cue may be high pitched and the next low pitched, etc. The 
information being conveyed is in the parsing of the topics. The voicemail system can then index the 
message so that the KAS tones form a series of navigable points within the message. In another 
embodiment, the association of audio cues with the sections of the message can be done as the message 
is retrieved, by either the voicemail system or the client telephone equipment (e.g., the cellphone). 
Retrieving the message -When a user acts to receive the message, the voicemail system can announce 
or otherwise convey (g o... with a numerical indicator on a cell phone display ) the existence, and the 
number of KAS delineated sections. This gives the user a sense of topics to be covered . That is, along 
with the date and time, and number of origin of the message, the number of topics can also be announced . 
Alternatively, the audio cues can indicate to the user the continuity of the topic . When the cues change, 
the topic has changed. 

Forward to the next topic -This invention enables a new voicemail feature - forward to the next topic. 
Each topic is delineated with a KAS and is navigable. Thus, the user can listen to the voicemait serially, 
or can listen to a few seconds of each topic, and skip to the next as necessary. This is particularly useful 
during replay of messages. When a user has listened to a message, and potentially missed noting a 
phone number for example, then if that phone number had been noted with a KAS, the user can navigate 
to it more easily. 

Actions on partial voicemaiis -With each section delineated, this invention enables new voicemail feature 
which act on each section individually. Any action that can be taken on a voicemail can be taken on a 
partial voicemail. The user C3n delete or forward partial voicemaiis. At the end or beginning of a KAS 
delineated topic, the user can elect to forward only the previous section or the upcoming section to a third 
party. This usefully allows the pertinent portion of a message to be forwarded, while allowing the user to 
retain privacy on the remainder of the message. Similarly, the user can choose to delete one or more 
delineated sections of a voicemail, retaining only those portions which he or she desires to save . 

Media Transformation: An audio cue can be used to minimize the effect of a media transformation from a 
non-audio source such as text. In one embodiment, each, paragraph separation is taken to be a KAS as 
above, and an audio cue ts mixed into the message. In another embodiment, an appropriate audio file is 
mixed with the resultant audio from the transformation, and Ihereby providing parallel information as to 
context In this invention, we are not inserting an audio sound or cue in-line in the message (e.g., a giggle 
sound in place of a smiley emoticon), but we are "conferencing" an appropriate audio file for part of, or all 
of, the message. An appropriate cue may be determined in several ways. If the originating source has 
supplied keywords, this can be used as a source of cueing. If the originating source has not supplied 
keywords, (e.g., today's e-mail systems), the transformation may select an appropriate noun in the first 
few sentences to use as a source of cueing. For example, if the first sentence of a paragraph reads "The 
wedding date has been set,", the appropriate audio cue may be church bells. If the sentence reads "The 
meeting was very productive" the appropriate audio cue may be papers rustling, and low conversation . 
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Other Information: Audio cues can provide additional contextual information, especially for 
transformations. For example, when machine language translation is involved, audio cues can be used to 
indicate degree of certainty in the translation. A background hum might indicate certainty of translation, 
with higher pitches indicating more certainty and lower pitches indicating less . Note that this may be 
usefully employed in an audio to audio transform, with voice recognition in the middle, or a text to speech 
plus language transform. Another example would be in the use of speaker identification to identfiy the 
speaker leaving a voicemail. The background tone can indicate the degree of certainty in the 
identification. Similar methods can be used to highlight passages in text as to degree of certainty, when 
for instance the text was obtained through voice recognition . Another example, in a text to speech 
system an audio cue could be used to indicate the color of the text (which may indicate the degree of 
importance, a change in topic, and etc), the background hum as described above could be applied in this 
case with the pitch indicating the level of importance, a different topic, or more simply a change in color 

Audio only messages: In another embodiment, we associate an audio emotional cue with an audio 
message. In this embodiment, the audio file is selected by the speaker to be mixed with the message, or 
inserted in the message directly without transformation . The file selected represents the context which the 
user wishes to convey. For example, for an angry message, the user may select quiet rolling thunder as a 
background. The user can select this effect before the message, during the message, or after the 
message is recorded with appropriate DTMF tones on the telephone keypad . This can also be done for 
parts of the message, as delineated wilh KAS tones. 

User selected audio background: In this embodiment, the speaker selects the audio background as part of 
the message being left. For example, the speaker may select from a number of offered backgrounds 
("top 40") what music is to play in the background of the message The music is mixed with the message, 
as the other audio cues ahove, In an alternate embodiment, the speaker's telephone client receives the 
selection request and communicates it to the voice mail system . If the speaker has unique or favorite 
music (for example as is used to provide ringing tone on a cell phone), and wishes to have this familiar 
identification (e.g., a speaker's "theme song") as part of the message being left, the telephone can 
communicate the music, or identify the music to the receiving voicemail system . 

Note that this embodiment enables a business model of merchandizing play out of popular music, via a 
subscription, or a pay per use of the music. In this model, it is expected that the speaker would pay for the 
music, as the receiver has no opportunity to accept or decline. This is consistent with the Calling Party 
Pays model standard in telephony. Again, the music may be transmitted from the phone, identified from 
the phone, or supplied by the voicemail system from a control indicator sent by the phone . Note that 
Sonera has a lucrative business of downloading ringing music to phones for 50 cents a crack; this is the 
equivalent for leaving messages. 

F 'j jfjH : Call flow leaving a message with delineation (including finding the phone number) 
LvMsg.PRZ 

Fin^n^2: Playing a voicemail which has delineation 
playvm.prz 

Fig ^jre^ - Leaving a message with context 
msgctxt.prz 

Figure 4: Histogram for the GUI; GUI is web driven for example 
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